🪣

Requester pays bucket

Motivation

Instead of requiring the PDP data uploader to also pay for its egress / retrievability, allow retrieval-interested parties to pay for the retrievals themselves. This frees up the role of the storage payer from that of the retrieval payer, albeit both can still be the same person.

Inspiration

Using Requester Pays general purpose buckets for storage transfers and usage - Amazon Simple Storage Service
Learn how to use Amazon Simple Storage Service (Amazon S3) to store and retrieve any amount of data from anywhere. This guide explains Amazon S3 concepts, such as buckets, objects, and related configurations, and includes code examples for common operations.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html

🔏Payer-authorized retrievals via FilCDN

Requester pays for retrievals · Issue #40 · filcdn/roadmap
Eng thoughts: Either we need a meta data set structure, where multiple meta data sets can be created over one real dataset. Or we need an 1:n mapping between data sets and subscriptions (one pays j...
https://github.com/filcdn/roadmap/issues/40#issuecomment-3183311008
  • Here’s the two potential mechanisms that could enable “requester pays for bucket”:
    • Adding mechanisms for users to top-up 3rd party account balance
    • Permitting users to enable CDN service for 3rd party data sets

    Note that neither of these mechanisms works in a
    pay-per-byte model where user retrieving the data would pay directly to
    SP or FilCDN for the data they’ve consumed, rather both of these
    mechanisms extend on the already existing contracts and infrastructure.

    Adding mechanisms for users to top-up 3rd party account balance

    Currently, users are permitted to top-up the balances of
    third-party accounts. However, there are no mechanisms in place to
    prevent the third-party user from operating the account linked to the
    CDN rails and withdrawing funds from it. To prevent this, the operator
    would need to either increase the streaming rate or implement a fixed
    lockup mechanism that prevents the third-party user from withdrawing the
    funds. This approach could be challenging, as it would grant users
    authority over third-party payment rails and potentially lead users to
    lose control over the deal.

    Permitting users to enable CDN service for 3rd party data sets

    The second solution is simpler, but it would require the
    introduction of a toggling mechanism for the CDN service in the FWSS
    contract beforehand. In the issue #125,
    I mentioned some mechanisms on how we could enable toggling the CDN
    service for a data set on and off. However, if there are no
    authorisation mechanisms, users could enable the CDN service for a
    third-party data set and pay the egress costs themselves. The downside
    of this solution is that deal termination becomes more complex.

Design

Schema

Retrievability can be added to a full data set (bucket), and doesn’t operate on individual piece level. See https://space-meridian.slack.com/archives/C08TVNKJV7C/p1755089591144709?thread_ts=1755084399.560739&cid=C08TVNKJV7C for discussion.

Problem

Now the main question is:

âť“

How to create a CDN deal for an existing data set

Status Quo

This is the current state of contracts:

  • CDN deals can only be made in the same transaction as making the storage deal. Either you add CDN when creating the storage deal, or you don’t
  • Because of this, CDN deals can also only be made by the data uploader themselves
  • For each data set, there can be exactly 0 or 1 CDN deals, as 2 CDN payment rail IDs exist as properties on the data set structure (one for cache miss, one for the CDN).

Contract updates

Either we update WarmStorageService, the main contract, to

  • Map 0:n CDN deals to each PDP deal, with arbitrary deal owners
  • Terminate CDN deals on storage deal termination

Or we create a separate contract, that implements this feature on top of WarmStorageService. This allows us to iterate more freely, at the cost of maintaing another contract with its own upgrade cycle, and another data source to sync with for the indexing backend.

In case of a separate contract, we need to make sure that termination of the WarmStorageService PDP deal also terminates the new contract’s CDN deal.